PipaSet and TEAS: A Multimodal Dataset and Annotation Platform for Automatic Music Transcription and Expressive Analysis Dedicated to Chinese Traditional Plucked String Instrument Pipa

Music information retrieval (MIR) is developing these years rapidly. As the fundamental MIR tasks, automatic music transcription (AMT) and expressive analysis (EA) are gaining momentum in both Western non-European music. However, annotated datasets for non-Eurogenic instruments remain scarce terms of quantity feature diversity so that general evaluations data-driven models on various tasks cannot be well explored. one most popular traditional plucked string Asia, which barely studied community, pipa has lots distinctive national local characteristics including fake nails, intrinsic pitch shift, rubato, as a high sophisticated playing techniques greatly enhance expressiveness. Our work aims to systematically clarify creation procedure dataset with audio, musical notation multiview video modalities Chinese solos. The use 4-track vibration signals captured by optical sensors paves path quality annotations. Furthermore, Expressiveness Annotation System (TEAS) was transparently implemented ensure scalability dataset. Three approaches this system were newly proposed evaluated paper. Finally, two AMT investigated series existing emerging enabled enumerated future exploration.